肌肉骨骼和神经系统疾病是老年人行走问题的最常见原因，它们通常导致生活质量降低。分析步行运动数据手动需要训练有素的专业人员，并且评估可能并不总是客观的。为了促进早期诊断，最近基于深度学习的方法显示了自动分析的有希望的结果，这些方法可以发现传统的机器学习方法中未发现的模式。我们观察到，现有工作主要应用于单个联合特征，例如时间序列的联合职位。由于发现了诸如通常较小规模的医疗数据集的脚之间的距离（即步幅宽度）之类的挑战，因此这些方法通常是优选的。结果，我们提出了一种解决方案，该解决方案明确地将单个关节特征和关节间特征作为输入，从而使系统免于从小数据中发现更复杂的功能。由于两种特征的独特性质，我们引入了一个两流框架，其中一个流从关节位置的时间序列中学习，另一个从相对关节位移的时间序列中学习。我们进一步开发了一个中层融合模块，以将发现的两个流中发现的模式结合起来进行诊断，从而导致数据互补表示，以获得更好的预测性能。我们使用3D骨架运动的基准数据集涉及45例肌肉骨骼和神经系统疾病的患者，并实现95.56％的预测准确性，效果优于最先进的方法，从而验证了我们的系统。

检测人对象相互作用对于全面了解视觉场景至关重要。特别是，人与物体之间的空间连接是推理相互作用的重要提示。为此，我们提出了一个用于人类对象相互作用检测的骨骼感知图卷积网络，称为SGCN4HOI。我们的网络利用了人类关键点和对象关键点之间的空间连接，以通过图卷积捕获其细粒的结构相互作用。它将此类几何特征与视觉特征和空间配置特征融合在一起，并从人类对象对获得。此外，为了更好地保留对象结构信息并促进人类对象的相互作用检测，我们提出了一种新型的基于骨架的对象关键点表示。 SGCN4HOI的性能在公共基准V-Coco数据集中进行了评估。实验结果表明，所提出的方法的表现优于最先进的姿势模型，并针对其他模型实现竞争性能。

Vision transformers (ViTs) are quickly becoming the de-facto architecture for computer vision, yet we understand very little about why they work and what they learn. While existing studies visually analyze the mechanisms of convolutional neural networks, an analogous exploration of ViTs remains challenging. In this paper, we first address the obstacles to performing visualizations on ViTs. Assisted by these solutions, we observe that neurons in ViTs trained with language model supervision (e.g., CLIP) are activated by semantic concepts rather than visual features. We also explore the underlying differences between ViTs and CNNs, and we find that transformers detect image background features, just like their convolutional counterparts, but their predictions depend far less on high-frequency information. On the other hand, both architecture types behave similarly in the way features progress from abstract patterns in early layers to concrete objects in late layers. In addition, we show that ViTs maintain spatial information in all layers except the final layer. In contrast to previous works, we show that the last layer most likely discards the spatial information and behaves as a learned global pooling operation. Finally, we conduct large-scale visualizations on a wide range of ViT variants, including DeiT, CoaT, ConViT, PiT, Swin, and Twin, to validate the effectiveness of our method.

预训练的视觉模型（例如，剪辑）在许多下游任务中显示出有希望的零弹性概括，并具有正确设计的文本提示。最近的作品不依赖手工设计的提示，而是使用下游任务的培训数据来学习提示。虽然有效，但针对领域数据的培训却降低了模型的概括能力，使其无法看到新领域。在这项工作中，我们提出了测试时间提示调整（TPT），该方法可以通过单个测试样本即时学习自适应提示。对于图像分类，TPT通过使用置信度选择最小化熵来优化提示，以便模型在每个测试样本的不同增强视图上都具有一致的预测。在评估对自然分布变化的概括时，TPT平均将零击的TOP-1精度提高了3.6％，超过了先前需要其他特定于任务的训练数据的迅速调整方法。在评估看不见类别的跨数据集泛化时，TPT与使用其他培训数据的最先进方法相当。项目页面：https：//azshue.github.io/tpt。

对抗性培训是生产模型的行业标准，对小对抗扰动具有鲁棒性。然而，机器学习从业者需要对自然发生的其他类型的变化具有强大的模型，例如输入图像的样式或照明的变化。输入分布的这种变化已经有效地建模为深度图像特征的平均值和方差的变化。我们通过直接扰动特征统计而不是图像像素来调整对抗性训练，以生产对各种看不见分布偏移的稳健的模型。通过可视化对抗特征，我们探讨了这些扰动和分布转变之间的关系。我们提出的方法，对抗批量归一化（ADVBN）是一种网络层，在训练期间产生最坏情况的扰动。通过微调对抗性特征分布的神经网络，我们观察到对各种看不见的分布转移的网络的改进的鲁棒性，包括风格变化和图像损坏。此外，我们表明，我们提出的对抗特征扰动可以与现有的图像空间数据增强方法互补，从而提高性能。源代码和预先训练的型号在\ url {https://github.com/azshue/advbn}释放。

Benefiting from the intrinsic supervision information exploitation capability, contrastive learning has achieved promising performance in the field of deep graph clustering recently. However, we observe that two drawbacks of the positive and negative sample construction mechanisms limit the performance of existing algorithms from further improvement. 1) The quality of positive samples heavily depends on the carefully designed data augmentations, while inappropriate data augmentations would easily lead to the semantic drift and indiscriminative positive samples. 2) The constructed negative samples are not reliable for ignoring important clustering information. To solve these problems, we propose a Cluster-guided Contrastive deep Graph Clustering network (CCGC) by mining the intrinsic supervision information in the high-confidence clustering results. Specifically, instead of conducting complex node or edge perturbation, we construct two views of the graph by designing special Siamese encoders whose weights are not shared between the sibling sub-networks. Then, guided by the high-confidence clustering information, we carefully select and construct the positive samples from the same high-confidence cluster in two views. Moreover, to construct semantic meaningful negative sample pairs, we regard the centers of different high-confidence clusters as negative samples, thus improving the discriminative capability and reliability of the constructed sample pairs. Lastly, we design an objective function to pull close the samples from the same cluster while pushing away those from other clusters by maximizing and minimizing the cross-view cosine similarity between positive and negative samples. Extensive experimental results on six datasets demonstrate the effectiveness of CCGC compared with the existing state-of-the-art algorithms.

To generate high quality rendering images for real time applications, it is often to trace only a few samples-per-pixel (spp) at a lower resolution and then supersample to the high resolution. Based on the observation that the rendered pixels at a low resolution are typically highly aliased, we present a novel method for neural supersampling based on ray tracing 1/4-spp samples at the high resolution. Our key insight is that the ray-traced samples at the target resolution are accurate and reliable, which makes the supersampling an interpolation problem. We present a mask-reinforced neural network to reconstruct and interpolate high-quality image sequences. First, a novel temporal accumulation network is introduced to compute the correlation between current and previous features to significantly improve their temporal stability. Then a reconstruct network based on a multi-scale U-Net with skip connections is adopted for reconstruction and generation of the desired high-resolution image. Experimental results and comparisons have shown that our proposed method can generate higher quality results of supersampling, without increasing the total number of ray-tracing samples, over current state-of-the-art methods.

Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning. However, we argue that these methods have overlooked two indispensable issues: 1) Boundary-bias: The annotated target segment generally refers to two specific frames as corresponding start and end timestamps. The video downsampling process may lose these two frames and take the adjacent irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new boundary frames also lead to the reasoning bias during frame-query interaction, reducing the generalization ability of model. To alleviate above limitations, in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames to enrich and refine the new boundaries. Specifically, a reasoning strategy is developed to learn the inter-relationship among these frames and generate soft labels on boundaries for more accurate frame-query reasoning. Such mechanism is also able to supplement the absent consecutive visual semantics to the sampled sparse frames for fine-grained activity understanding. Extensive experiments demonstrate the effectiveness of SSRN on three challenging datasets.

Representing and synthesizing novel views in real-world dynamic scenes from casual monocular videos is a long-standing problem. Existing solutions typically approach dynamic scenes by applying geometry techniques or utilizing temporal information between several adjacent frames without considering the underlying background distribution in the entire scene or the transmittance over the ray dimension, limiting their performance on static and occlusion areas. Our approach $\textbf{D}$istribution-$\textbf{D}$riven neural radiance fields offers high-quality view synthesis and a 3D solution to $\textbf{D}$etach the background from the entire $\textbf{D}$ynamic scene, which is called $\text{D}^4$NeRF. Specifically, it employs a neural representation to capture the scene distribution in the static background and a 6D-input NeRF to represent dynamic objects, respectively. Each ray sample is given an additional occlusion weight to indicate the transmittance lying in the static and dynamic components. We evaluate $\text{D}^4$NeRF on public dynamic scenes and our urban driving scenes acquired from an autonomous-driving dataset. Extensive experiments demonstrate that our approach outperforms previous methods in rendering texture details and motion areas while also producing a clean static background. Our code will be released at https://github.com/Luciferbobo/D4NeRF.

Deploying reliable deep learning techniques in interdisciplinary applications needs learned models to output accurate and ({even more importantly}) explainable predictions. Existing approaches typically explicate network outputs in a post-hoc fashion, under an implicit assumption that faithful explanations come from accurate predictions/classifications. We have an opposite claim that explanations boost (or even determine) classification. That is, end-to-end learning of explanation factors to augment discriminative representation extraction could be a more intuitive strategy to inversely assure fine-grained explainability, e.g., in those neuroimaging and neuroscience studies with high-dimensional data containing noisy, redundant, and task-irrelevant information. In this paper, we propose such an explainable geometric deep network dubbed as NeuroExplainer, with applications to uncover altered infant cortical development patterns associated with preterm birth. Given fundamental cortical attributes as network input, our NeuroExplainer adopts a hierarchical attention-decoding framework to learn fine-grained attentions and respective discriminative representations to accurately recognize preterm infants from term-born infants at term-equivalent age. NeuroExplainer learns the hierarchical attention-decoding modules under subject-level weak supervision coupled with targeted regularizers deduced from domain knowledge regarding brain development. These prior-guided constraints implicitly maximizes the explainability metrics (i.e., fidelity, sparsity, and stability) in network training, driving the learned network to output detailed explanations and accurate classifications. Experimental results on the public dHCP benchmark suggest that NeuroExplainer led to quantitatively reliable explanation results that are qualitatively consistent with representative neuroimaging studies.